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I INTRODUCTION 



A. PREFACE. 

A primary goal of this thesis is to analyze the Longbow 
HELLFIRE system, the Hardware in the Loop simulation 
methodology and the statistical properties of lot acceptance 
testing for the Army Material Command (AMC) . This methodology 
will be compared to the existing HELLFIRE II "Fly to Buy" 
methodology that has been in effect for the last several 
years . 

B. SYSTEM DESCRIPTION. 

1. Missile System Description. 

The Longbow HELLFIRE Modular Missile System (LBHMMS) 
is an integral part of the AH-64D helicopter weapon system, 
which is designed to defeat multiple armored ground targets 
and several selected air targets, through the process of radar 
acquisition targeting and ground engagement (see Figure . 
The LBHMMS consists of: 

• Longbow HELLFIRE Modular Missile. 

• Longbow HELLFIRE Launcher. 

• Longbow Missile Container. 

• Longbow Training Missile. 

• Radome Environmental Cover. 
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Figure 1 Longbow HELLFIRE System Description 
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In addition, a missile telemetry sub-system will be 
designed and used during flight testing of several of the 
Engineering and Manufacturing Development (EMD) Longbow 
missiles^ . 

Tactical Missile Description. The LBHMMS consists of an active 
Millimeter Wave (MMW) radar guidance section mated to a 
HELLFIRE II missile bus. The Longbow missile warhead is a 
High Explosive Anti-Tank (HEAT) weapon that employs a radar 
aided inertial guidance system. The bus consists of a warhead 
section, a propulsion section, and a control section (see 
Figure #2) . The LBHMMS is capable of either a Lock-On-Before- 
Launch (LOBL) Mode or a Lock-On-After-Launch (LOAL) Mode which 
describes the two methods in which a target can be engaged by 
the Longbow Missile. These are the major components and 
functions that will be referred to through out this thesis. 

2. Launch Platforms. 

The AH-64D Helicopter is the primary launch platform 
envisioned for the LBHMMS. The LBHMMS will provide the AH-64D 
and other airborne launch platforms with a fire-and- forget 
capability for engaging targets handed over from the Longbow 
Fire Control Radar (FCR) or its functional equivalent systems, 
such as the Tactical Air Designation System (TADS) , Airborne 
Target Handoff System (ATHS) and the Integrated Helmet and 



^ For additional information concerning the Longbow 
HELLFIRE program and points of contact see Appendix A. 
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Figure 2 



Longbow HELLFIRE Tactical Missile 



Display Site System (IHADSS) . Future variants may also 
include a ground launch platform in addition to the airborne 
platforms described above. 

C . BACKGROUND . 

For the past several decades, the U.S. Army has required 
virtually all existing tactical missile systems, be purchased 
only after a representative sample of missiles from each lot, 
the production lot, has been successfully tested to confirm 
specified perfoimance requirements. The purpose of such lot 
acceptance testing plans is to provide a qualitative 
assessment of missile system reliability for the entire lot 
and to provide an accept/reject decision rule for the entire 
lot of missiles. Under this procedure, the contractor only 
delivers and gets paid for, those lots that pass this 
acceptance test. 

The "Fly to Buy" acceptance methodology, by its very 
nature, requires that these sample missile systems be fired in 
a scenario that represents the actual or anticipated 
requirement that the missile system would be likely to 
encounter under actual battlefield conditions. This 
inevitably results in the missile being destroyed while 
verifying that it is functioning properly. 

Until recently, the technical complexity of the missile 
system and the cost associated with performing "Fly to Buy" 
testing was not large enough to cause rethinking of the 
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process. However, increasing costs of missile systems and 
declining defense resources are forcing a re-evaluation of 
this type of destructive lot acceptance testing. Other more 
cost-effective testing methods must be found to determine the 
quality of missile lots. 

One possible alternative to this problem is to assess the 
missile's quality through tests involving computer 
simulations. This process known as Hardware -in -the -Loop 
(HWIL) testing, attempts to simulate the flight of the missile 
without actually requiring it to be fired or destroyed. In 
this case the actual sample missile would be attached to a 
computer via a cable tied to as many of the key components of 
the system as safely practical. A computer simulation of an 
actual acceptance test scenario could then be run that 
exercises the missile in a manner similar to an actual live 
firing against a specified target. The computer collects the 
data, analyzes them and assesses the results in terms of the 
lot acceptance criteria. 

No specific Government methodology exists that addresses 
the prospect of simulating missile fights via this HWIL 
process for lot acceptance. Currently there is no other Army 
millimeter wave HWIL facility which can accommodate "live" 
missile rounds. This would require the construction of a 
separate facility that could perform this type of testing. 
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D . PURPOSE . 

The purpose of this thesis is to explore and evaluate the 
current Longbow HELLFIRE Lot Acceptance Plan against the 
conventional "Fly to Buy" methodologies to determine what 
advantages have been gained using a HWIL simulation for this 
process and what still remains to be accomplished in order to 
use this new process effectively. 

E. APPROACH. 

Because the Longbow HELLFIRE Quality Assurance Lot 
Verification Test (QALVT) plan is a one of a kind test plan 
that still is in its draft form, it will be necessary to 
consider past plans that preceded it to fully understand and 
appreciate the advantages it presents. 

The approach used in conducting this analysis involves 
first providing the background information leading up to the 
establishment of an HWIL simulation method. This will be 
discussed and provided in Chapters I & II. Next the 
traditional "Fly to Buy" methodology will be discussed in 
Chapter III, to provide the reader with a basis of 
understanding of the key requirements involved in conducting 
prior lot acceptance testing and the whole lot acceptance 
procedure in general. This will also serve to outline the 
development of the HWIL simulation method. 

The current draft HWIL Simulation Plan will be introduced 
in Chapter IV, and its differences highlighted from its 
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predecessor the "Fly to Buy" methodology. In Chapter V the i 
HWIL Simulation method is analyzed to quantify its benefits 
and deficiencies. A conclusion to the analysis is provided in 
Chapter VI. Finally, in Chapter VII, suggested modifications 
and alternatives (including possible contractual language) of 
the HWIL methodology are discussed. 



8 



II THE NEED FOR HARDWARE IN THE LOOP (HWIL) ACCEPTANCE 

TESTING 

A. BACKGROUND. 

In FY 1992, the Air- to-Ground Missile System (AGMS) 
Project Office, which includes HELLFIRE, HELLFIRE II, and 
Longbow HELLFIRE, solicited a proposal from the United States 
Army Test and Experimentation Command, Redstone Technical Test 
Center (USATECOM RTTC) to develop an alternative method for 
performing lot acceptance testing on Longbow HELLFIRE Missiles 
during Low- Rate Production and subsequent Full -Rate 

Production. Prior lot acceptance testing had been performed 
exclusively through the use of "Fly to Buy" (FTB) methodology, 
where a lot sample of 4 to 10 missiles per month were flight 
tested at Eglin AFB to determine acceptance or rejection for 
the remainder of the missiles in each lot. In the intervening 
years between the production of HELLFIRE II and the 
development of Longbow HELLFIRE, there has been a significant 
rise in the unit cost of each missile round. The conventional 
"Fly to Buy" lot acceptance program would require considerable 
additional resources. An HWIL simulation based testing 
program will reduce the lot acceptance testing costs. 
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B. UNITED STATES ARMY MISSILE COMMAND (USAMICOM) POLICY # 

702-5. 

The Commanding General of USAMICOM has directed that 
weapon system acquisition planning will consider the principal 
items expressed in Quality Assurance Lot Verification Testing 
(QALVT) before making a final determination as to what type of 
requirements will be included in the final missile lot 
acceptance plan. The contents of this directive was set forth 
in MICOM POLICY No. 702-5, dated 16 May 1990. 

The application of this policy is based on three principal 
decision factors: 1) unit cost of missile less than or equal 
to $ 200k, 2) production rate of at least 50 units per month, 
3) estimated QALVT less than 5% of the procurement cost (see 
Figures # 3, 4, & 5, U.S. Army QALVT decision Criteria)^. 

C. ASSUMPTIONS AND TRADE-OFFS. 

The Air to Ground Missile System (AGMS) Project Office and 
USATECOM RTTC have performed an analysis of the perspective 
factors and trade-offs involved in implementing a HWIL lot 
acceptance program versus a traditional "Fly to Buy" (FTB) 
program. 

This analysis considered the two alternatives available to 
Longbow HELLFIRE (LBHF) as well as the historical data 
provided by the earlier HELLFIRE I. In all cases, it was 
assumed that all FTB flight testing would be conducted at 

^ US Army Missile Command Policy #702-5, 16 May 1990. 
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QUALITY ASSURANCE LOT VERIHCATION TESTING (QALVT) 

CRITERIA FOR MEETING "DIAMOND A" (WHICH ALLOWS PROGRESSION 
TO THE NEXT STEP IN THE QALVT LOGIC DIAGRAM) 

1 . QUAUTY PROGRAM IN ACCORDANCE WITH MIL-Q-9858. 

2. GOVERNMENT APPROVED STATISTICAL PROCESS CONTROL (SPC). 

3. SUPPUER RATING PROGRAM IN ACCORDANCE WITH MIL-STD-1535. 

4. LAST PRODUCT ORIENTED SURVEY (POS) SCORED GREATER THAN 8.0. 

5. NO CATEGORY 1 QUALITY DEHCIENCY REPORT (QDR) RECOVED IN 
THE LAST 12 MONTHS. 

6. SIX OR LESS CATEGORY 2 QDRs RECOVED IN THE LAST 12 MONTHS. 

7. NO MAJOR HARDWARE DESIGN CHANGES (SUCH AS PRODUCT 
IMPROVEMENT PROGRAMS) SINCE THE START OF QALVT STEP 1 . 

8. TEN OR FEWER MAJOR OR CRITICAL RFDs/RFWs DURING LAST 12 
MONTHS. 

9. NO MAJOR CHANGES IN FACIUTIES, TOOUNG, TEST EQUIPMENT OR 
PRODUCTION PROCESSES SINCE THE START OF QALVT STEP 1 . 

10. RECOVING INSPECTION YIELDS GREATER THAN 93 PERCENT DURING 
THE LAST SIX MONTHS. 

1 1 . IN-PROCESS INSPECTION YIELDS GREATER THAN 93 PERCENT DURING 
THE LAST SIX MONTHS. 

12. INTERNAL CONTRACTOR QUALITY AUDIT DATA ARE AVAILABLE TO 
THE GOVERNMENT AND CONTRACTOR AGREES TO UNANNOUNCED 

GOVERNMENT AUDITS. 

13. MISSILES ARE WARRANTED. 

14. NO POST-ACCEPTANCE FLIGHT FAILURES OF WARRANTED MISSILES. 

NOTE: ITEMS 1 THROUGH 14 ABOVE ARE PROVIDED FOR GUIDANCE AND 
MAY BE MODIFIED OR CHANGED THROUGH CONTRACT NEGOTIATION. 
Figure 5 MICOM Policy 702-5 Decision Criteria (cont.) 
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Eglin Air Force Base (EAFB) , Alabama. The primary factors 
inthe development of the assumptions involved in this analysis 
process were as follows: 

Unit Costs: 

• HELLFIRE cost per missile $ 20k 

• Longbow HELLFIRE (LBHF) cost per missile $ 300k 

@ Low Rate Initial Production (LRIP) . 

Test Programs: 

• HELLFIRE Lot Acceptance Testing: (FTB) @ 4-10 

missiles per month. 

• Longbow HELLFIRE (LBHF) Lot Acceptance Testing: 

(FTB) @ 4-10 missiles per month. 

• Longbow HELLFIRE HWIL Simulation: 4 live fires per year 
and up to 20 missiles simulated flight testing per month. 

Facilities : 

• Facility for the HWIL Simulation: $ 5.8 million (a one 
time cost) . The actual building and 
instrumentation. 

Labor : 

• Personnel needed to run this facility for one year: 
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estimated at approximately $0.8 million. 



With these factors in mind, a trade-off analysis was 
performed using costs for four missiles with the results 
portrayed in Table 2-1 Production/Acceptance Analysis^. 

The total savings projected under a HWIL simulation lot 
acceptance test program would represent cost savings of $13.55 
million per year over a conventional "Fly to Buy" program. 
Even including the $5.8 million necessary for providing for 
the HWIL simulation facility, the payback period under this 
assumption would be less than one year. 



Table 2-1, PRODUCTION/ACCEPTANCE ANALYSIS. 
(in $ millions per year) 



! 


HELLFIRE FTB 


LBHF FTB 


HWIL SIM. 


1 MISSILE COST^ 


$1.15 


$14.40 


$1.20 


1 EAFB SUPPORT 


$0.72 


$0.72 


$0.06 


RTTC SUPPORT 


$0.53 


$0.53 


$0.44 


SYS. SIM SUPPORT 


$0.00 


$0.00 


$0.40 


TOTAL 


$ 2.40 


$ 15.65 


$ 2.10 


Notes: 1. HWIL SIM. 


missile costs 


represent the cost of hook- 



up and processing of sample missiles through the 
computer test facility and returning them to their 
original configuration. 



The HELLFIRE FTB column indicates approximately what AGMS 
pays per year for the HELLFIRE I "Fly to Buy" program, and 
also indicates where the true cost growth in FTB actually 
lies, namely, missile cost. This total amount includes 



^ Johnson, J., Memorandum for AGMS, "All Up Round Test 
Facility", 18 April 1994. 
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missile cost, range support, and pre- flight support. The 
Longbow HELLFIRE "Fly to Buy" column indicates what the cost 
would be if the AGMS Office implemented a similar "Fly to Buy" 
program with just four missiles per month, even though this 
method might require up to ten missiles to complete. 

D. HWIL SIMULATION REQUIREMENTS. 

The requirements imposed on the HWIL facility by AGMS are 
as follows: 

• Functional checkouts of hardware and software inter- 
operability of the missile. 

• Test must provide a confidence equal or better than FTB, 
that missile lots are good or bad. 

• Must be non-destructive. 

• Must perform testing at the All-Up-Round (AUR) level. 

• Must consider environmental conditioning. 

• Number of required missiles must be significantly lower. 

Testing at the All -Up-Round level, refers to conducting a 
test at the complete, assembled system level. AUR would not 
allow components of the system to be disassembled to 
accommodate testing. 

The facility required for HWIL testing would also have to 
have specific requirements. These would include the following 
items : 

• The HWIL facility be located at Redstone Arsenal 
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(RSA) . 



• Capable of Vibration & Environmental testing. 

• Capable of handling a Longbow HELLFIRE All Up Round (AUR) . 

• Meet safety requirements. 

Provide cost effective operation. 

E. THE SIMULATION/TEST ACCEPTANCE FACILITY. 

The Simulation/Test Acceptance Facility (STAF) is the 
Hardware- in- the Loop facility being developed at RSA by the 
U.S. Army Test and Evaluation Command (USATECOM) Redstone 
Technical Test Center (RTTC) and the U.S. Army Missile Command 
(USAMICOM) and System Simulation and Development Directorate 
(SSDD) . This facility is the only facility in the U.S. 
capable of performing nondestructive HWIL tests on "live" 
production and developmental missiles at the millimeter wave 
frequencies. A diagram of the facility is shown at Figure 6. 
This facility is currently under construction at RSA on the 
north-eastern side of Test Area #1. This facility includes 
the following equipment, buildings and hardware: 

1 . Bunker . 

This structure is approximately 2,000 square feet that 
features a test item room, a test chamber and a computer room. 
The test item room contains the compact missile test set and 
the environmental chamber for conditioning missile rounds 
prior to testing. The test chamber contains the missile under 
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18 



test, the Three Axis Rotational Flight Simulator (TARFS) , 
anechoic chamber, and the target generator horns of the 
millimeter wave target simulator. The computer room contains 
the target generation equipment and instrumentation, telemetry 
instrvimentation, simulation computer, and the data analysis 
terminals (for inert rounds) . There will be a roll-up door in 
this structure facing down range that will allow actual target 
acquisition and tracking, if desired, down the long axis of 
the TA-1 range. This will allow the actual tracking of a 
variety of real world target vehicles with actual production 
missiles, at ranges up to approximately 6km. 

2. Target Generator. 

The target generator system will intercept the 
millimeter wave signal transmitted by the missile, delay the 
signal in time to simulate the range to the target, tap delay 
the signal to simulate target range extent, place proper 
doppler shift on the signal to compensate for relative 
movement and return the signal to the missile in real-time on 
a pulse for pulse basis. 

3. Three Axis Rotational Flight Simulator (TARFS). 

The TARFS will provide a mounting structure for the 
missile in the test chamber. It will additionally provide 
real-time missile flight motion in pitch, yaw, and roll to 
simulate the missile's actual fly out to the target along its 
own trajectory. The missile fin position will be monitored 
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and fed back into the simulation in order to control the TARFS^ 
position. j 

f 

4. Telemetry. 

All missile and STAF data will be collected and time 
tagged to allow for a real time "quick look" and post mission' 
data analysis. 

5. The Simulation Computer System. 

The simulation computer system will consist of two 
computer systems; a control computer and a modeling computer. 
The control computer will run the 6 Degrees Of Freedom (6 -DOF) 
program, facility control software, perform inputs and outputs 
to the modeling computer and interacts with the user. The 
modeling computer simulates the complex signature of the 
target subject using parallel processing techniques. 

6. Compact Missile Test Set. 

This test set will perform the majority of the open 
loop testing and characterization of each missile round prior 
to entering into simulation testing. This test set will 
expose obvious flaws or defects in each missile prior to this 
testing and prevent wasting valuable simulation time on 
defective rounds. 
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Ill THE EXISTING "FLY TO BUY" METHODOLOGY 



A. INTRODUCTION. 

The purpose of the "Fly to Buy" Methodology is to 
determine within an acceptable level of statistical 
confidence, that the Government is purchasing (in lot size 
quantities) a product that is reasonably reliable and defect 
free. In the case of missile procurement, this is usually 
accomplished after destructively testing the lot sample and 
comparing the results against pre-established acceptance 
criteria. The sample size required to accomplish this task is 
related to the lot size and the confidence interval required. 
Since most missile systems tend to be quite expensive, this 
can be very costly in the long run. Consequentially, a 
sequential sampling plan is usually developed that reduces the 
average number of missiles tested yet satisfies the 
requirements. These plans are divided into several stages 
which specify a certain quantity and an acceptance/rejection 
criteria. Often these plans, depending on the lot size and 
confidence required, will specify that testing be continued 
and that another stage of testing be conducted up to a pre- 
determined quantity. At the end of the final stage, a 
decision will be made to accept or reject the lot and no other 
testing will be continued for the lot. 
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B. THE HELLFIRE LOT ACCEPTANCE PROGRAM. 



1. Purpose. I 

The purpose of the HELLFIRE Lot Acceptance Program is 

to provide confidence that HELLFIRE missiles meet the 
specified performance and reliability requirements of the 
production contract^. 

2. Bachgroxind. 

a. San^ling Plan Characteristics . 

This plan is intended to provide a method for 
performing lot acceptance by prescribing an Acceptable Quality 
Level (AQL) for the production process. This AQL is used to 
determine the sample size for the sampling inspection. The 
AQL is the largest allowable percent of defective items of a 
satisfactory process average. This sampling plan and its AQL 
are chosen in accordance with the risk assumed, as defined in 
the contract. Thus, the AQL is the designated value of 
percent defective for which lots will be accepted most of the 
time by the sampling procedure to be used. The sampling plan 
is usually arranged so that the probability of rejecting a lot 
when the process average is at the designated AQL value is not 
larger than a preassigned small value (eg. 0.10) . This is the 
producer's risk. The producer's risk may vary slightly with 
the sample size. 



^ The HELLFIRE II Modular Missile System QTILVT Plan, 
USAMICOM, July 1989, is the primary source of information for 
this chapter. 
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b. Operating Characteristics Curves. 

In military standard lot acceptance testing 
documents, the operating characteristic curves (OC curve) for 
normal inspection, indicate the percentage of lots or batches 
which may be expected to be accepted under the various 
sampling plans for a given process quality. These OC curves 
are normally shown for single sampling; curves for double and 
multiple sampling are matched as closely as practicable. The 
OC curves shown for AQLs greater than 10.0 are based on the 
Poisson distribution and are applicable for defects per 
hundred units inspection; those for AQLs of 10.0 or less and 
sample sizes of 80 or less are based on the binomial 
distribution and are applicable for percent defective 
inspection; those for AQLs of 10.0 or less and sample sizes 
larger than 80 are based on the Poisson distribution and are 
applicable either for defects per hundred units inspection, or 
for percent defective inspection (the Poisson distribution 
being an adequate approximation to the binomial distribution 
under these conditions) . Tabulated values, corresponding to 
selected values or probabilities of acceptance (P^, in 
percent) are given for each of the curves shown. These tables 
are also given for tightened inspection processes. An example 
of a typical OC curve is shown at Figure 7. 
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Figure 7 Typical OC Curve for Lot Acceptance Purposes 





3. HELLPIRE Lot Acceptance Definitions and Procedures. 



Missile - A missile shall consist of one LBHF High 
Explosive Anti-Tank (HEAT) missile as defined by Missile 
Specification (MIS-SPEC) MIS-44430 or MIS-42560. 

Test Article - The test article will be a HFLB HEAT 
production missile which has been selected for sample testing. 

Lot - A lot shall consist of the production quantity 
for a given month which has been produced under the same 
processes. In all cases, missile lot sizes shall be subject 
to the constraints of homogeneity with respect to changes in 
the processes, configuration changes, deviations or waived 
hardware, etc. , which may materially affect performance of the 
end item. The Government reserves the right to combine 
monthly build lots when necessary/appropriate. 

Lot Sample - A lot sample is the quantity of ten 
missiles randomly selected from a lot build of 100 missiles or 
more which will be environmentally conditioned and then tested 
for the purpose of lot acceptance or rejection. Each lot 
sample of ten missile is further divided into three subgroups 
consisting of lot sample test article numbers 1-4, 5-8, and 9- 
10 respectively. 

4. Sampling Procedure. 

As indicated above, in the case of HELLFIRE, ten 
missiles will be randomly selected from each lot build, and 
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shall be delivered to Redstone Arsenal (RSA) , Alabama, for 
environmental conditioning and then shipped to Eglin Air Force 
Base (EAFB) for the actual missile flight testing. The 
Government then shall accept or reject the entire lot of 
missiles (which is anticipated to be roughly one month's 
production) based on the following sequential sampling plan; 



Table 3-1, SEQUENTIAL SAMPLING PLAN. 



No. Msls Tested 


No. Msl 


Failures 


(Cumulative) 




Accept 


Continue 


Reject 




Lot 


Testina 


Lot 


4 


0 


1 


2 


8 


1 


2 


3 


10 


2 




3 



In addition to the ten missiles selected for testing. 



two contingency missiles shall be selected from each lot. 
These two contingency missiles will be used in the event that 
any of the ten test articles are damaged or otherwise 
unsuitable for testing. This will prevent disrupting the test 
process if one or two missiles have to be substituted for 
these reasons. 

Smaller samples shall be selected from lots of less 
than 100 units per month (a distinct possibility in the case 
of diminishing procurement funding or program stretch outs) . 
The minimum number of missiles to be selected under these 
conditions is as follows in Table 3-2 below. 
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Table 3-2, SMALL SAMPLE SIZES. 



LOT SIZE 


LOT SAMPLE SIZE 






1-10 


1 


11-25 


2 


26-49 


3 


50-99 


5 



The Government shall identify to the contractor 



missiles (by serial number) which have been selected for QALVT 
testing at the time the missiles have successfully completed 
final inspection at the production facility. 

After these missiles are identified to the contractor 
as test articles, the contractor will then seal the missile 
containers with contractor furnished seals, and shipped to RSA 
for testing. The contingency missiles may be held at the 
contractor facility or shipped with the test articles to RSA. 

The missiles selected for lot sample testing shall be 
counted as a portion of the deliverable quantity of the 
applicable accepted lot. If any seals or containers are 
broken, the Government reserves the right to require 
replacement of the missile in the lot sample with a 
contingency missile. The remainder of the lot of missiles 
which have not been selected for testing at RSA, shall be 
packed in missile containers and placed in secured storage 
until completion of the lot acceptance testing, for which they 
are part, is complete. The serial number of each missile for 
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each lot shall be provided to the Government at the time it is 
placed into secured storage. 

Once a selected test article has been identified to 
the contractor, no further tests, adjustments, or repairs 
shall be performed except as authorized by the Government. 
Test articles rejected on arrival at RSA shall be 
dispositioned by pre- flight nonconformance procedures. The 
contractor shall test missiles at RSA on the HELLFIRE Missile 
Compact Test Set (HMCTS) in accordance with procedures 
specified in the QAVLT Plan and perform pre- flight check out 
at EAFB on the HELLFIRE Compact Firing Test Set. The 
Government reserves the right to omit any portion of the lot 
acceptance test, including environmental tests, of any or all 
of the sample missiles. After Government inspection at RSA, 
test articles shall be stored in a Government security locked 
area . 

5. Flight Testing. 

After Government inspection and acceptance procedures 
at RSA have been completed and prior to shipment to EAFB, the 
lot sample missiles will be subjected to Captive Flight 
Vibration (CFV) and one subgroup will be environmentally 
conditioned at RSA, in accordance with the lot acceptance plan 
and the applicable missile specifications. 

Upon completion of these tests and environmental 
conditioning at RSA, the missiles will be transported to EAFB 
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for pre- flight and firing per Table 3-3. 



TABLE 3-3, TEST FIRING MATRIX, 



LOT # 


TEST ART 


PROFILE 


MODE 


RANGE 


COND. 


6n+l 


1 


INDIRECT LOW 


LOAL 


L 


AMB 


6n+l 


2 


INDIRECT HIGH 


LOAL 


L 


AMB 


6n+l 


3 


DIRECT 


LOBL 


M 


AMB 


+ 1 


4 


r T F CT 


liCBTi 


M/B° 


A1*IB 








XJ\^ XD JLl 


l a/ w 


11^ 


6n + 2 


1 


INDIRECT HIGH 


LOAL 


L 


COLD 


6n+2 


2 


INDIRECT LOW 


LOAL 


M 


COLD 


1 6n+2 


3 


DIRECT 


LOAL 


S 


COLD 


€n+2 


4 


DIRECT 


LCEL 


M/6° 


COLD 


6n+3 


1 


INDIRECT LOW 


LOAL 


S 


HOT 


6n+3 


2 


DIRECT 


LOBL 


S/8° 


HOT 


6n+3 


3 


DIRECT 


LOAL 


M 


HOT 


€n+3 


4 


irroiRECT HIGH 


LOAL 


M 


HOT 


6n + 4 


1 


DIRECT 


LOAL 


M 


AMB 


6n+4 


2 


INDIRECT HIGH 


LOAL 


M 


AMB 


6n+4 


3 


INDIRECT LOW 


LOAL 


S 


AMB 


6n+4 


4 


DIRECT 


LOBL 


S,^0° 


?J4E 


1 6n+5 


1 


DIRECT 


LOBL 


M 


COLD 


6n+5 


2 


INDIRECT HIGH 


LOAL 


M 


COLD 


6n+5 


3 


INDIRECT LOW 


LOAL 


L 


COLD 


€n+S 


4 


DIRECT 


LOBL 


S/8° 


[ COLD : 


6n+6 


1 


DIRECT 


LOBL 


M 


HOT 


6n+6 


2 


DIRECT 


LOBL 


M/8° 


HOT 


6n+6 


3 


INDIRECT HIGH 


LOAL 


L 


HOT 


6n+6 


4 


INDIRECT LOW 


LOAL 


L 


HOT 



Where n = 0, 1, 2, 3 Lock- on can be either 

before or after launch (LOBL or LOAL) and Range which will 
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be at short (up to 2km) , medium (2-5) and long (+5) , can 
include an 8° offset. Offset is the deviation from the 
straight line trajectory that the missile selects (either 
right or left) to help it to distinguish the actual target 
from possible background clutter. Conditioning (COND.) 
represents the environmental temperature conditioning either 
at ambient (AMB) , cold (-45 F) or hot (+145 F) . 

The procedures used to conduct the actual firing at 
Eglin AFB will be controlled by that installation. Missiles 
will be transported, inspected and prepared for firing. 

This will involve conducting a test readiness review (TRR) 
to insure that all the test requirements, range support, 
instriimentation, and safety considerations have been 
satisfied prior to actual missile firing. 

The objective of this specified missile firings is to 
mix two ranges, all modes, offsets, with varying degrees of 
difficulty on each block of the four firings so that all 
missile specifications will eventually be tested when this 
table is completed. There are a few limitations. They are: 
1) Only two firing ranges (distances) can be instrumented on 
the firing range at one time, and 2) All firing from any one 
lot will be performed at the same temperature conditioning for 
all test articles. The Government may require the contractor 
to fire at any of the conditions in the matrix above or any 
combination of conditions that are in accordance with the 
appropriate Military Specification (MIL-SPEC) . Prior to 
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firing, the Government and contractor shall confirm that all 
firing equipment and associated range equipment are in proper 
working condition and calibrated. 

6. Scoring. 

All test articles shall be scored as either a: 1) 

Success, 2) No Test, or 3) Failure, in accordance with the 
following criteria. 

a. Pre- flight Non conf orma.n ce: 

Any nonconformance of a test article or to Test 
Design Package (TDP) requirements discovered during inspection 
(including nonconformances discovered prior to, during, or 
after environmental testing of missiles at RSA or pre- flight 
checkouts at EAFB) shall constitute cause for failure of the 
test article and may be cause for lot rejection. The 
following procedure will apply in determining the extent of 
further testing on the subject lot: 

• Testing of a missile shall be stopped at the point any 
failure or nonconformance is discovered and a preliminary 
analysis shall be made by the contractor and submitted to 
the Government within 24 hours. 

• The Government will then determine whether this missile 
will continue through test and be fired or whether it will 
be failed and returned to the contractor for failure 
analysis and future production. 
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• If a missile exhibits a pre-flight nonconformance and the 
Government elects to continue testing that missile, that 
particular pre- flight nonconformance shall be scored a "no 
test " . 

• If the Government determines from the failure analysis 
that the pre- flight nonconformance will or is uncertain it 
will have an impact on the flight test parameters, the 
missile will be scored a safety or reliability failure as 
appropriate . 

Based on the preliminary analysis of the pre- 
flight nonconformance, the Government and contractor may 
jointly elect to continue testing of the lot. A determination 
to continue shall not relieve the contractor of the 
responsibility to correct the failure condition in all 
missiles, as deemed necessary by the Government, 
b. No Test 

A no test may stem from a variety of causes and 
sources related to the missile under test or circumstances 
surrounding these tests. They will include areas resulting 
from: 

• A malfunction of equipment other than the test article or 
proven post launch environments which causes the missiles 
to experience conditions outside of those specified in the 
Missile Specification (MIS-SPEC) . 
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• Failures induced by Government personnel error or failures 
induced by errors of those authorized to act for the 
Government, either deliberate or inadvertent, or operation 
beyond prescribed limits. 

• In the event that the contractor aborts a missile launch 
and the subsequent analysis proves that the missile did 
not experience an anomaly, the missile will be scored a 
"no test". The contractor shall pay all retest costs and 
the Government will notify the contractor of the 
acceptance or rejection of the lot within 45 additional 
calendar days beyond the time identified in the lot 
acceptance plan. 

• Although a failure did not occur, the missile displays a 

property not previously observed and is, therefore, of 
special interest. It is anticipated that, if left in the 
test sequence, this missile will pass all tests and when 
launched will fly successfully and hit the target within 
the required accuracy thus fulfilling QALVT requirements. 
If the missile is permitted to remain in the test sequence 
all the way through to a successful launch, the 
opportunity to determine the cause of this newly observed 
property will be conducted by the appropriate range 
personnel. If this occurs prior to launch, a 

Government/contractor decision shall be made regarding 
further testing of this missile. 
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c. Failures. 



Failure ; Any malfunction (not meeting the no test 
criterion) which would preclude the missile from performing 
within the requirements of the MIS-SPEC will be categorized as 
either a safety or reliability failure as follows: 

Safety failures: A lot shall be rejected as a result 

of one pre- flight nonconformance or missile firing that 
results in a condition that may be hazardous or unsafe as 
defined in the Technical Design Package (TDP) or as defined 
below. A safety failure will have occurred if: 

• A warhead detonates prior to the minimum safe arm 
distance from the launcher of 150 meters. 

• An inadvertent launch occurs which is traced to a 
missile malfunction. 

• There is a delay (hang fire) of more than five 
seconds between the initiation of the firing 
signal and the ignition of the rocket motor. 

• Any event that would endanger the launching 
helicopter (e.g. major debris thrown in the 
area by the missile upon launch) . 

Reliability Failure: Accept/reject criteria for 

reliability shall be attributed based on the total number of 
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failures. Accept/reject nuinbers shall be as specified in the 
sampling plan (see Table 3-1, Sequential Sampling Plan, pg 
26) . A reliability failure shall have resulted when one or 
more of the following occurs: 

• A missile misses the target. A target miss shall 
have occurred if an intact missile does not meet 
the accuracy requirements as stated in the 
corresponding MIS -SPEC. 

• The missile impacts the ground prior to impacting 
the target . 

• The warhead fails to detonate in accordance with 
the timing requirements of MIS -SPEC. 

• The warhead detonates during flight due to a 
missile malfunction and is not otherwise scored a 
safety failure. 

• A no- fire occurs. A no- fire is defined as failure 
of the missile to fire following application of 
the firing signal. 

• The missile flies a profile other than that 
preset . 

• A pre- flight nonconformance determined to be a 
reliability failure in accordance with the QALVT 
Plan. 
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d- Success. 



Success : All missile flights not scored as a no- i 

test or failure will be scored as a success. All available 
test data and flight hardware debris shall be made available 
to the contractor as soon as practical; visual examination of 
impact debris and recovery of the maximum amount of missile 
hardware debris, that can safely be handled, shall be allowed^ 
prior to destruction of the remainder of the hazardous ‘ 
components (if deemed feasible by the Explosive Ordinance | 
Disposal unit (EOD) ), contingent on approval by the on-site | 
EOD representatives. Scoring of missile launches as success, | 
failure, or no- test will be accomplished by the Government. 1 

I 

7. Acceptance/Rejection Criteria. 

Acceptance or rejection of a lot is based on pre- 
flight inspection and tests at RSA and EAFB and flight 
acceptance tests on a sample of missiles randomly selected by 
the Government from each lot and scored in accordance with the 
lot acceptance plan. In addition missiles will not be 
accepted until all waivers, deviations. Engineering Change 
Proposals (ECPs) , or other issues affecting a lot are resolved 
to the Government's satisfaction. 

Upon accepting a lot of missiles, the contractor shall 
deliver to the Government all remaining lot sample missiles, 
(including contingency missiles) not flight tested. Missiles 
which have been disassembled/repaired in failure analysis and 
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have been subjected to lot acceptance captive flight and 
vibration testing will be made functional, by completing a 
functional baseline test at the lot temperature designated at 
RSA and accepted for retirement at RSA. 

8. Resubmlttal Criteria. 

Lots which fail to pass the acceptance criteria may be 
resubmitted by the contractor unless an alternative plan of 
action for the lot acceptance of missiles is agreed upon 
between the Government and contractor. Failure analysis will 
be conducted and deficiencies corrected from the initial 
submittal prior to the resubmittal as described above. A 
failed lot may be resubmitted a maximum of two times. All 
scoring and rejection criteria will remain the same except for 
adjustments made in the quantity of test articles required for 
decreased lot sizes. 

9. Contractor Liability. 

When a lot sample fails the acceptance criteria and is 
rejected by the Government or when a lot meets the acceptance 
criteria and one or more failures occur, the contractor shall 
perform failure analysis and shall submit a detailed report 
describing the deficiencies in performances which resulted in 
rejection or failure and the action necessary to correct and 
prevent recurrence of the deficiencies. The contractor's 
report shall be prepared in accordance with the data item 
reporting requirements contained in DI-RELI- 80253 . 
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In the event of a failed lot, additional lots shall 
not be submitted for acceptance until corrective action has ii 
been approved by the Government. Also, testing may be 
suspended at the Government's option on any test articles 
located at RSA or EAFB. 

10. Responsibility for Retesting Rejected Lots. ' 

In the event lots are rejected, any and all expenses ' 
associated with failure analysis, replacement hardware, ' 
rework, reinspection, packaging, packing, handling, | 

transportation, and retest shall be borne by the contractor. [ 
The foregoing expenses are inclusive of the cost of ’ 
replacement of missiles. Government Furnished Equipment (GFE) , 
cost of range operations, environmental conditioning, and | 

I 

additional failure analysis. With respect to retest costs 
assuming a resubmittal of 4 missiles, the contractor shall 
reimburse the Government in the amount of $4,000 for each 
missile environmentally conditioned at RSA, and $10,000 for 
each missile flown in each resubmitted lot test sample. At 
the contractor's option and expense, the remaining test 
missiles from a rejected lot may be returned to him. 
Replacement missiles will be in the same quantity expended in 
initial test, retest, and those rendered non -deployable due to 
environmental conditioning less the ten (10) missiles 
allocated for each lot tested in the acceptance plan and 
contingency missiles that replace "no test" missiles. 
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11. Responsibilities. 

a. Contractor Representative . 

The contractor representative is responsible for 
technical support effort in observing, analyzing, and 
supporting program plans and procedures to meet program 
milestones and objectives. Additionally, he will coordinate 
with other supporting agencies to insure mission support 
requirements are fulfilled in a timely manner. The contractor 
shall provide a single individual at RSA and EAFB who shall 
act as the point of contact for all decisions related to local 
test activities. The contractor may participate in all test 
mission planning sessions. Mission Readiness Reviews (MRR) , 
data reviews, and/or briefings. 

The contractor shall provide all missile round 
information and simulation results required to describe 
expected flight performance of the missile. This shall 
include nominal flight trajectories and expected three sigma 
variation bounds of this trajectory. The contractor may 
monitor the launch and shall assist the Government in solving 
any problem areas which may arise due to missile malfunction. 

In case of a suspected missile "failure" or "no 
test" the following activities shall be performed: 

• The contractor shall perform a failure analysis on any 
test article which are suspected failures or "no tests". 
Results of this analysis and recommendations for further 
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activities shall be prepared in a Failure Analysis Report 
in accordance with data item reporting requirements 
contained in DI-RELI- 80253 . 

• The contractor shall prepare a Corrective Action Plan in 
accordance with DI-RELI- 80254 for all failures which 
result in the rejection of a missile lot. This plan shall 
describe all activities and actions planned by the 
contractor to solve the missile failure problem. 
Government approval of the Corrective Action Plan is 
required prior to implementation. 

In the case of a rejected lot, the contractor 
shall develop and prepare a plan for lot resubmission in 
accordance with DI-RELI- 80254 indicating the scheduling and 
activities to be performed to resubmit the lot to the 
Government for acceptance. Government approval of this plan 
is required prior to implementation. 

Jb. U.S. Air Force. 

U.S. Air Force - Range Support Contractor & Range 
Support Representative; 

The U.S. Air Force Support contractor and 
representative is responsible for overall flight range 
operations to include Data Acquisition and Control System 
operation, test instnamentation, data collection and 
reduction, missile ready storage and delivery, environmental 
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chamber operation, range security, range safety, conduct of 
the countdown from upload to launch, missile recovery 
operations, target complex operations, data and documentary 
camera operation, laser spot data, and geodetic surveys of 
missile impact points. 

U.S. Air Force - 46th Test Wing - Test 
Engineer /Coordinator : 

The Air Force test engineer/coordinator will be 
responsible for overall Air Force coordination, management, 
and scheduling of resources to support missions on Range C-72, 
the Longbow HELLFIRE flight test range. 



U.S. Air Force Range Systems Directorate 
Representat ive : 

The Range Systems Directorate formally controls 
all test procedures, instrumentation, and software 
configuration. No change or variation in procedures or 
configuration will be made without prior written approval of 
the Range System Directorate engineering representative. All 
questions concerning range instrumentation or facility 
performance, readiness, and utilization will be directed to 
the Range Systems representative. 

No information or data generated at the facility 
will be released without the representative's Quality 
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Assurance review. All requests for assistance or support from 
the Range Operation and Maintenance (OScM) contractor shall be 
made through this representative. No direction of the O&M 
contractor shall be made by the missile contractor or other 
support personnel . 
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IV THE HARDWARE IN THE LOOP (HWIL) SIMULATION METHOD 



A. INTRODUCTION. 

As in the case of many areas of Government regulation, lot 
acceptance has several underlying requirements that must be 
addressed regardless of the methodology adopted to perform the 
overall controlling action. This is certainly true for the 
Longbow HELLFIRE missile. Since several aspects of HWIL will 
not differ much from those areas already identified in the 
"Fly to Buy" methodology, only those areas that differ will be 
addressed. Therefore, the reader can assume that all other 
areas not specifically addressed will remain the same. 

B. LONGBOW HELLFIRE HWIL SIMULATION QALVT. 

1. Purpose 

The purpose of the Longbow HELLFIRE Quality Assurance 
Lot Verification Test (QALVT) program is to provide a cost 
effective means of determining the acceptability of Longbow 
HELLFIRE production missiles while still meeting the specified 
missile performance and reliability requirements stated in the 
contract^ . 



^ The Longbow HELLFIRE Modular Missile System, Low Rate 
Initial Production QALVT Plan, USAMICOM, 23 May 1994, is the 
source of information used throughout this chapter. 
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2. The QALVT Plan 



The Longbow HELLFIRE QALVT plan provides a method to | 
subject a random sample from a lot of Longbow HELLFIRE 
missiles to a nondestructive, environmentally conditioned, 
dynamic, mechanical, and functional test through the use of a 
Hardware in the Loop Simulation that depicts an electronic j 
target similar to those that are anticipated to be encountered 5 
under future battlefield conditions. The resulting miss 
distance would be the primary performance measure. 

3. Definitions. 

Missile - A "missile" shall consist of one Longbow 
HELLFIRE HEAT missile as defined in the MIS. j 

Lot - A lot shall consist of the production quantity 
for a given month which has been produced under the same j 
conditions utilizing the same processes. ' 

Lot Sample - A quantity of fifteen missiles randomly I 
selected from a lot which will be tested in accordance with 
the QALVT Plan. 

I 

4. Lot Seunpllng Procedure. ■ 

Instead of the ten missiles required for FTB, fifteen 

missiles shall be randomly selected from each lot and 
delivered to RSA. These fifteen missiles shall have been 
previously tested on special inspection equipment, to be 
determined (TBD) , and shall have passed all inspections and 
tests. The Government may require the identification of all 
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of the contingency missile (s). 

While at RSA, the fifteen missiles will be tested in 
the Simulation/Test Acceptance Facility (STAF) . The 
Government will then accept or reject the entire lot of 
missiles based on the following simulation sampling plan 
criteria described in Table 4-1 below. 



TABLE 4-1, SIMULATION (HWIL) SAMPLING PLAN. 



No. Missiles 
Tested 


Number of 


Missile 


Failures 




Accept 


Continue 


Reject 


6 ■ 


0 


1 


2 1 


6 


1 


2 


3 


3 


2 




3 



Procedures for selection and transportation of these missiles 
to RSA is basically the same as with the "Fly to Buy" method. 
Unless physically damaged in transportation, these missiles 
will be determined to be suitable for testing once they have 
arrived and been inspected by the Government representative at 
the STAF facility. And as with FTB method, the missiles 
selected for lot sample testing will be counted as a portion 
of the deliverable quantity of the applicable accepted lot. 

5. Test Sequence and Pass Fall Criteria, 
a. Incoming Inspection. 

There is no pass/fail requirement for this test 
other than the Government visually inspecting the missile for 
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signs of damage due to shipment prior to the actual testing. 

b. Perfoirm Portable Missile Test Bench (PMTB) Tests. 

This is a system pre- check performed by the 

contractor utilizing portable test equipment. The actual 
portable test equipment that will be used to perform this test 
is anticipated to be fielded in conjunction with the missile 
system. 

c. Perform Limited Vibration Test. 

This limited vibration test will be designed to 

I 

simulate some of the vibration that the missile system would 
be exposed to if it had been actually fired. The levels of 
this vibration are to be determined in the future. 

I 

d. Missile Preparation. 

Missile preparation will consist of removing the 
precursor door and installing an electrical harness called a 
"pigtail", in order to connect the internal missile guidance I 
electronics to the HWIL simulator. Additionally, it is 
anticipated that an external pneumatic gas line will be 
connected to the fin mechanism to actuate it during the 
simulation to demonstrate that the fin system is responding 
properly to guidance commands. However, in order to 
accurately assess this fin response, it would be necessary to 
replicate the aerodynamic loading of the fins. This process 
is a very difficult task to perform in a laboratory 
environment. Because of the difficulty involved, the 
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decision as to whether or not this will be attempted has not 
been made at this time. 

e . Envxronmen tal Condi ti oning . 

All missiles selected from a lot shall be shipped 
to Redstone Arsenal, but only one subgroup will be initially 
environmentally conditioned. The environment conditioning 
will be for 24 hours. The next subgroup from the lot will not 
begin environmental conditioning until the previous subgroup 
has been tested and the need to test the second subgroup 
established. The contractor may select the temperature for 
the first lot; thereafter, the temperature conditioning 
sequence will typically continue as follows: 



TABLE 4-2, MISSILE ENVIRONMENTAL CONDITIONING. 



LOT NO. 


SUBGROUP 


SAMPLE SIZE 


TEMP. 


n + 1 


1 


6 


AMBIENT 




2 


6 






3 


3 




n + 2 


1 


6 


COLD 




2 


6 


(-45 F ) 




3 


3 




n + 3 


1 


6 


HOT 




2 


6 


(+145 F ) 




3 


3 




Where n = 1, 
2, 3 



Once environmental conditioning has been 
completed, the missile will be removed from the chamber and 
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installed on a CARGO® table with shroud. The CARGO table is 
a special test fixture that reduces the effect of unwanted 
vibration during tests and allows the entire test stand to be 
moved within the STAR facility test cell. 

f. Closed Loop Electronic Tests. 

Six flight test scenarios will be simulated by the 
STAF"^. Four scenarios will be against Stationary (S) target 
at ranges from 1.5 kilometers (km) to 7.5km. Two of the 
stationary target scenarios will be conducted at ranges of 1.5 
km and 2.49 km in Terminal Track Acquisition (TTA) and 
Terminal Track (TT) modes, both with Precision (P) Hand over 
(P H/0) . One will Lock- on Before Launch (LOBL) and the other 
will Lock-on After Launch (LOAL) . The remaining two 
stationary target scenarios will be conducted at ranges of 6 
km and 7.5 km in modes Pre-Terminal Track Acquisition (PTA) , 
Pre- Terminal Track (PTT) , TT and Reduced Radar Cross Section 
(RCS) . Both will employ lock on mode LOAL and P H/0. 

Two scenarios will be against moving (M) targets 
at 6km range and will employ LOBL lock on mode. One will 
employ P H/0 with Moving Target Acquisition (MTA) (spotlight) , 
PTT and TT modes while the other employs Non- Precision (NP) 
H/0 with MTA (scan), PTT Ground Evaluation (GE),& TT modes. 



® CARGO is the name of the manufacturer who produces this 
test fixture. 

Specific tasks that will be performed within the 
closed loop tests are described in Appendix B. 
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These scenarios are outlined in Table 4-3. 

The requirement for success is that each missile 
shall be tested three times (runs) for each scenario. A 
missile must pass two mns out of these three for each 
scenario in order to be a success. 

The miss distance requirement for each run is 2 x 
Circular Error Probable (CEP) requirements. 



TABLE 4-3, CLOSED LOOP ELECTRONIC TESTS. 



SCENARIO 


TARGET 


RANGE 


LOCK ON 


H/0 


MODES TESTED 


1 


S 


1 . 5km 


LOBL 


P 


TTA, TT 


2 


S 


2 . 491an 


LOAL 


P 


TTA, TT 


3 


M 


6km 


LOBL 


P 


MTA spotlight 
PTT, TT 


3 


M 


6)on 


LOBL 


NP 


MTA scan 
PTT, GE, TT 


5 


S 


61cm 


LOAL 


P 


PTA , PTT , TT 


6 


S 


7.5km 


LOAL 


P 


PTA, PTT, TT 



The general layout of this closed loop test can be 
seen at Figure 8 on the following page. 
g. Open Loop Tests. 

Open loop testing concerns checlcing other areas of 
missile system performance that cannot be performed as part of 
the closed loop segment of the HWIL simulation itself®. 
These areas involve conducting the following tests and tasks: 

® The specific tasks that will be performed within the 
open loop tests are described in Appendix B. 
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• RF Transit Output Test - in accordance with 
Longbow HELLFIRE MIS - SPEC-455 84 , to within 
minus 4 dB. 

• Basic Fin Response Test - This is part of the 
PMTB test that will be done prior to closed 
loop tests. 

• IMS Performance - Move CARGO table and measure 
gravity- relative position + 10%. 

• Remove shroud and missile from CARGO table. 

• Remove pigtail and re-install precursor door. 

• Perform PMTB test. 

Recertification 

Recertification will be in accordance with 
procedures that will be determined at a later date, but 
involve insuring that the missile (after completing the lot 
acceptance test) is placed back in to the same condition it 
was in prior to the start of testing. 
h. Resubmittal Criteria. 

Lots which fail to pass the initial testing may be 
resubmitted by the contractor unless an alternative plan of 
action for the lot acceptance of missile is agreed upon 
between the contractor and Government. 

Failure analysis shall be conducted and 
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deficiencies corrected from the initial submittal prior to 
resubmittal as outlined in the contractor's plan for lot 
resubmission which will be prepared following the procedures 
described in DI-RELI- 80254 . A failed lot may be resubmitted 
a maximum of two times. 
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V ANALYSIS 



A. INTRODUCTION. 

In the process of analyzing the Longbow HELLFIRE HWIL 
Simulation plan and methodology, several key issues regarding 
the basic structure and nature of the HWIL methodology come to 
light. The first one involves the very manner in which the 
test plan is structured. The issue here being whether or not 
the concept of Hardware in the Loop testing needs to follow 
the traditional structure typified by the FTB methodology or 
even the current HWIL version. This is because many of the 
reasons for the structure are no longer important to the 
outcome. Several factors need to be discussed in view of this 
very pertinent statement and for several reasons: 

1) Since HWIL does not require destructive testing of the 
missile, or its components, the traditional cost saving 
procedures described in earlier versions of the HELLFIRE 
QALVT or similar sampling procedures need not necessarily 
apply to the HWIL simulation. 

2) The quantity of test articles needed to perform HWIL 
testing need not be limited to just that of a minimum 
quantity necessary to provide a desired statistical 
confidence that a particular lot is good or bad. 
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3) In order to insure that this method actually provides an ' 
accurate assessment of the true nature of the product, a 
validation of the system/methodology must be conducted to 
substantiate, that the process will provide the desired 
information without an increase in the error probability or 
that some other undesirable result would occur. 

4) Actual missile flights will still be conducted to gather 
additional test data and to provide a quality check on the 1 
entire system (to include those areas that cannot be tested h 
using HWIL, such as the warhead detonation train) . The role 
that these missile flight tests will play in the lot t 
acceptance plan needs to be resolved. 

5) How should the contract requirements for lot acceptance 

I 

testing be altered to take greater advantage of the cost 
efficiency that HWIL simulation could provide? 

With these factors and considerations in mind, it seems 
prudent that the first step in analyzing the HWIL process, is 
to quantify/qualify all of the inputs and outputs of the FTB 
and HWIL simulation processes, and that of the Longbow 
HELLFIRE missile system that will be tested, and then compare 
these results to the factors and considerations described 
above. Finally these results should be compared with the 
Draft HWIL Simulation QALVT Plan to see what modifications 
would be warranted. This process should help provide a basis 



54 



of where or how the current plan could be modified to 
accommodate better approaches. 

Next a method for validating the HWIL simulation should be 
identified or developed. 

B. THE TWO APPROACHES. 

1. HELLFIRE (Fly To Buy). 

The original "Fly To Buy" methodology was designed to 
provide an acceptable level of protection for the Government 
and an acceptable level of risk to the contractor. This 

acceptable level is based on the anticipated reliability of 
the system under inspection. In the case of the Longbow 
HELLFIRE system, this missile system reliability was 

determined to be 0.94, and the acceptable quality level (AQL) 
was placed at 0.92. Considering the high cost of the 
HELLFIRE missile, the plan was structured to achieve a 

decision as to individual lot acceptability at the end of the 
least amount of destructive testing. In the case of earlier 
HELLFIRE missile systems, this led to the development of the 
4-4-2 multi-stage plan described in Chapter III. 

This plan can be evaluated via an OC curve that 

presents the inherent probability that the lot of missiles 
will be accepted based on the test results of the sample of 
missiles tested. The OC curve for the original FTB plan is 
depicted below at Figure 9. 
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J ^ 1’ sampling plan 




O 4-0 + 6-1 P 10-2 



Figure 9 HELLFIRE OC Curve 



56 



Since the prohibitive cost involved in conducting this 
type of testing makes it unsuitable, it is safe to assume that 
except for the AQL, no other factors are pertinent to the 
overall QALVT issue. 

2. The Longbow HELLFIRE QALVT Plan. 

The Longbow HELLFIRE QALVT Plan, for reasons similar 
to the those for the HELLFIRE FTB and the desire to take 
advantage of HWIL ability to exercise the missile under all 
six of the scenario conditions, has a 6-6-3 multi-stage plan. 
This plan has the proviso that each missile will be run 
through each test scenario three times and that each missile 
is required to pass at least two of the three scenario trials 
in order to pass any given scenario. Failure is still 
determined by whether or not each missile successfully passed 
all six scenarios and whether or not the overall sample 
successfully passed the acceptance test. This statement also 
holds true regardless of where the acceptance determination is 
made, either during the first, second or third stages of 
testing. The specific characterisitics of the 6-6-3 plan and 
the resulting OC curve, by stage, can be seen in Figure 10. 

The other major features of the HWIL QALVT Plan, that 
need to be discussed are the scenarios themselves. Because 
all of the Longbow HELLFIRE missiles must be run through each 
of the six scenarios, one might be concerned that one 
particular scenario might be more difficult than the others to 
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C 6-3 SAUPLINC PLAN 




D 6-0 + 12-1 O 15-2 



Figure 10 Longbow HELLFIRE HWIL OC Curve 
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pass. If this were true, than the acceptability of the entire 
lot might often be based on whether or not this scenario could 
be passed by the required number of missiles. To better 
understand this issue MICOM conducted a simulation study to 
determine to what extent each scenario was more difficult than 
another. The results of this simulation are depicted below in 
Table 5-1, Closed Loop Electronic Tests. 



TABLE 5-1, CLOSED LOOP ELECTRONIC SIMULATION TESTS 



Scenario 


Results* 


#1 


+ 1 


#2 


(-19) 


#3 


0 


#4 


0 1 


#5 


(-21) 


#6 


+ 9 



* Results are given in millimeters. 



Each scenario was conducted 100 times and the Results column 
in the table displays the difference in millimeters between 
each scenario and scenario #3. The results are given in 
Circular Error Probability (CEP) . Since the intended target 
of the Longbow HELLFIRE missile is a tank roughly 2 meters 
wide by 7 meters long by 2 meters high, the maximum deviations 
indicated in this table between each scenario is not very 
significant . 
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3. Comparison. 

When comparing the two plans it appears that the HWIL 
QALVT Plan places the contractor at greater risk then the 
original FTB. This comparison is given in Table 5-2. 



TABLE 5-2, COMPARISON OF PROBABILITY OF ACCEPTANCE. 





4-4-2 FTB Plan 


6-6-3 HWIL Plan 


STAGE #1 


0.78 


0.70 


STAGE #2 


0.92 


0.84 


STAGE #3 


0.98 


0.94 


OVERALL TOTAL: 


0.97 


0.92 



The information depicted in Table 5-2 was based on the 
assumption that the missiles being tested in either of the 
plans, do in fact, meet the contract system reliability 
requirement of 0.94^. 

In fact, the contractor has noted this increased risk 
and has contacted the Longbow HELLFIRE Program Office to 
register his concern. He has stated that he is being placed 
at greater risk than his previous exposure under former 
HELLFIRE contracts utilizing FTB. The PMO's response has been 
to take the issue under consideration until such time as the 
entire HWIL simulation plan solidifies to a point where a 
better determination on this issue can be made. Both plans 

^ Dick, R., "Binomial Sampling Plans Compaison", Brief, 
January 1990. 
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satisfy a requirement to accept the lot with probability 
greater then 0.92 when actual missile reliability is 0.94. 
Except for the testing cost factor ratio of approximately 7:i 
in favor of HWIL, both plans would be acceptable for 
Government use . 

The only other major area of comparison is the 
quantity of the sample size. In both cases the quantity used, 
was driven by the desire to make a determination of lot 
acceptability after the least amount testing and with the 
fewest number of missile test articles involved. The 
structure of each plan is basically the same except for the 
additional requirement, under the HWIL QALVT, that each 
missile must complete two out of three runs successfully for 
all six scenarios. This reason might give the impression of 
a significant difference. However, the likelihood of a 
missile passing one of the six scenarios, given it has passed 
one or more of the other scenarios is larger than it would be 
if this information were not available. That is, the outcomes 
of the scenario's tests are not statistically independent. In 
addition, the repeated three trials for each component are not 
statistically independent because it is the same missile under 
test in all three trials. This is based on the premise that 
HWIL testing involves testing only the electronic hardware of 
the guidance and control systems and the system software that 
run these components. It stands to reason that if these two 
components perform reliably the first time, they are more than 
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likely to continue to perform appropriately for all the 
remaining trials within that scenario. 

Thus, the trials and scenarios are not truly 
independent of one another as is assumed in the plan. This 
independence feature was used to compute the acceptance 
probability of 92%. If this factor of dependence is accounted 
for in a fairly conservative manner, the true acceptance 
probability is closer to 94%. While this number is closer to 
the FTB number, it may still represent additional risk to the 
contractor. The extent of this additional risk cannot be 
assessed without actual test data. 

The real issue here is not the number of scenarios, 
number of trials, or pass/fail criteria, but the structure of 
the test in general. Many different test structures can be 
designed and their probability of lot acceptance computed as 
can be seen in Table 5-3 Sampling Plan Comparison. 

However, what comes to light in the analysis of these 
test sampling plan schemes, is that they all are based on the 
assumption of multi-stage lot acceptance criteria, which 
assumes that there is a underlying desire to make a 
determination after the least cmiount of testing, because of 
the high cost that would normally be involved if this were 
true. This is not the case for HWIL simulation tests. As 
described earlier in Chapter II, the STAF facility can test 
about 20+ missiles a month at the same cost to do the 15 
missiles in the plan. 
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TABLE 5-3, SAMPLING PLAN COMPARISON. 



Sampling Plan 
1 (N - Acc - Re j ) 


Overall Probability of Acceptance 
given a Reliability of 0.94 


20-2-3 


0.885 


10-1-2 


0.882 


9-0-2 

9-1-2 


0.762 


8-0-2 

8-1-2 


0.IS$ 


• - 0 - 2 

• - 1 - 2 


0.836 


8-0-2 

8-1-2 


0.872 


** 6 - 0 - 2 ** 
6-1-3 
6-2-3 


0.92 (0.937) 


7-0-2 

7-1-3 

7-2-3 


0.891 


8-0-2 

8-1-3 

8-2-3 


0.858 


Fly To Buy 




4-0-2 

4-1-3 

2-2-3 


0.971 



Note; The **6-0-2** indicating the current test scheme. 
The 6-6-3 multi-stage HWIL plan still requires all 15 missiles 
to be delivered to the Government since there is no way of 
determining before hand, at what stage the decision will be 
made . 
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C. PLIGHT TESTING. 



1. Introduction. 

Flight testing beyond EMD and LRIP for the Longbow 
HELLFIRE missile will consist of firing 4 production missiles 
from static launchers, at Eglin AFB, FI. One missile will be 
chosen from each batch of lot samples for each quarter year. 
These missiles will have already passed their particular lot 
acceptance test in the STAF facility prior to there selection f 
for firing. Once selected they will be transported to EAFB | 
and prepared for firing against one of the six scenarios used j' 
in the lot acceptance plan. This flight testing is expected [ 
to provide a quality check on areas of the missile system not 
tested under either of the earlier open and closed looped 

I 

testing performed at RSA. These areas will include: i 

1) Rocket Motor and ignition device. 

2) Electronic Safe and Arm device (ESAF) . 

3) Warheads and detonation train. 

4) The Missile on board batteries. 

5) Other Flight related Components not tested via HWIL 

testing. 

Flight testing will also serve as a quality check and 
source of additional information on the HWIL Simulation 
process since all flight test missiles will be drawn from HWIL 
specimens. However, other than verifying that a test missile 



64 



hits or misses the intended target, there is little means of 
effectively gathering useable data from these flight test 
missiles. None of these missiles will be electronically 
instrumented or outfitted with telemetry devices that could 
transmit guidance, trajectory, and flight data back to a 
ground station for post flight analysis. The only data 

gathering mechanism is the high speed filming from range 
tracking devices and the point of impact. 

While the need for this testing is not being 
questioned, some very real and important questions do exist 
that could potentially adversely effect the HWIL lot 
acceptance process. 

2. Question Areas. 

a. Flight Test Significant. 

As mentioned above, one missile a quarter will be 
test flown at EAFB. At this time there is no particular 
plan/guideline to cover or evaluate these flight test 
missiles. The sole evaluation criterion will be whether the 
missile hits or misses the intended target. Because these 
missiles are drawn from samples that have already passed the 
HWIL simulation, any failures during these flight tests will 
generate a series of unanswerable questions that will be 
difficult or impossible to resolve. Such questions as: 

1) What specifically went wrong with the missile? 

2) Should it have passed the HWIL Simulation and then 
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failed to hit the actual target; i.e., is there 
a potential defect in the simulation testing? 

3) What do these flight test results indicate about 

the rest of the missiles in this particular production lot? 

4) What needs to be fixed and how? 

The first of these questions "What specifically 
went wrong?", may never be answered. Since there is no! 
instriamentation, only obvious failures such as the rocket i 

motor not igniting or the warhead failing to detonate could be ji 

( 

investigated with any certainty. Less obvious problems’ 
involved in target misses may never be resolved. 

"Should it have passed the HWIL simulation and! 

i 

f 

then missed the target?". Because post flight data are'; 

I 

limited, this question may never be answered satisfactorily. 
In fact, this particular question will likely serve to 
generate additional speculation in other areas such as 
transportation and mishandling. 

The third question presents quite a major dilemma, 
because the minimiun mimber of missiles necessary to answer 
this question is the seume quantity that is indicated in the 
former FTB plan; namely four missiles per lot. Since there 
will be only one missile fired from one of the three lots 
accepted during that period, no meaningful inference can be 
made about other missiles in the lot using the one flight 
test . 
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Finally, the last question on the list represents 
the fall out from the other three. This question can only be 
answered after decisions are made relating to the other three 
questions. However, if some prior thought is not given to 
these questions before hand and a course of action taken, then 
this area will have to be dealt with after the fact when it 
will be potentially more difficult and costly to resolve, 
jb. HWIL Feedback. 

It is understood that prior to any actual lot 
acceptance testing being conducted in the STAF facility, the 
process used by this facility will have to be validated to a 
certain level. Once that level is achieved, it will become 
difficult to improve on the process, because the source of 
data necessary to accomplish this activity; namely, telemetry 
instrumented test missiles will no longer be available. It 
stands to reason that the more information one has concerning 
a population of missiles (data base) the greater fidelity the 
modeling and simulation of that missile will be. The same 
holds true here. But as of now, there are no plans to build 
additional telemetry rounds, since these missiles are 
configured with a telemetry package in place of a warhead. 

Any retro- fitting of production missiles is not 
desirable from two perspectives, 1) it will not be truly 
representative of the rest of the lot it was produced from, 
once retro- fitted, and 2) the cost involved makes it 
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impractical . 



Another alternative available to the Government to ; 
improve the HWIL simulation process, would be to allow the 
firing of several of the missiles that have been determined to 
be not acceptable by the closed loop test only. They could be 
taken to the flight test range and fired for the purpose of 
gathering additional information and insight into the process 
itself. Such a program would help to verify the accuracy of 
the simulation. This testing program may not be advisable for i 
safety reasons, but the ability to confirm the good as well as ! 
the bad missiles would be a valuable feedback tool to improve | 
the process as a whole. 

D. VALIDATION AND CERTIFICATION 

1. Background and Requirements. 

HWIL simulations have been used on several occasions I 

I 

to help make lot acceptance determinations. Such systems as , 
the Army's TACMS and Pershing missiles, and the Navy's 
Tomahawk Land Attack Missile (TLAM) have all used HWIL 
simulations in some form, but not as the basis for a lot 
acceptance process. In each case the validation of these 
simulations poses an all together different problem for each 
system for different reasons. Validation is concerned with 
the fidelity of the model and its accurate portrayal of the 
real life event. The procedures required to accomplish this 
validation are not always the same because system 
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characteristics and requirements differ so radically from one 
to another. The validation process must be attacked on a case 
by case basis tailored for the unique characteristics of the 
system and the end product. 

As of now, there is no set plan to accomplish this 
task, but several factors will be key to its successful 
completion. They are 1) the determination of the accuracy of 
the simulation to its real life counterpart, 2) the inherent 
reliability of the simulation to function properly and 
accurately portray the proper environment without inducing 
additional errors due to defects, anomalies or modeling errors 
into the process. In both of these cases, any factor that 
would decrement the outcome of this process from the real life 
environment that the missile would be subjected too, could be 
perceived by the contractor as additional risk to him and be 
a basis for an increase in costs and missile price. For 
example, suppose the simulation's accuracy was determined to 
be 95% faithful to the actual system's characteristics. What 
does this 5% difference represent to the two parties involved? 
Will it impact on the determination of the acceptability of 
the lots under inspection? 2) If the simulation software were 
deemed to work properly and produce an accurate result 99% of 
the time, the 1% of the time that it did not function properly 
could be perceived by the contractor as an additional 1% of 
risk he was assuming to perform lot acceptance. This would 
represent 1% more lots that might not be accepted by the 
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Government costing him additional rework and handling that may 
not be justified. Of course, the opposite case is also 
possible. The HWIL simulation process might be passing 
defective missiles. 

No verification plan has been formulated or drafted to 
date, but several of these problems have been recognized by 
the facility developers. The mechanism requiring each sample 
missile to pass only two of three runs of each scenario is an 
example of an attempt to reduce the prospect of a simulation 
induced error effecting the overall acceptance of any 
particular lot. 

These examples indicate an area of the Longbow 
HELLFIRE QALVT plan that has not been fully covered. One area 
that deserves additional attention pertains to what will 
happen when a HWIL simulation problem occurs during the 
pursuit of lot acceptance testing. Potential concerns in this 
area would center around the necessity of restarting that 
lot's testing, because it may not be determined if the problem 
detected in the missile under test went undetected in all 
prior missiles. If so, this would tend to invalidate prior 
testing on missiles for that lot. This would likely require 
retesting. 

2. A Possible Validation Model. 

As mentioned above, no specific lot acceptance 
simulation has been through the validation process to date. 
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However, there have been simulations that have been validated 
and certified for purposes very close to that which would 
serve the Longbow HELLFIRE system very well as a possible 
model for their certification program. 

This system is the Navy's Tomahawk Land Attack Missile 
(TLAM) , which used a computer simulation called the 
Interpretive Simulation Program No. 13 (ISP 13.0). 

The ISP 13.0 simulates the TLAM and the environment in 
which it operates. It focuses on Block III and Pre -Block III 
Operational Flight Software (OFS) execution from power-up to 
missile impact. 

The plan that the Naval Air Warfare Center, Weapons 
Division (NAWCWPNS) used to certify their simulation involved 
a concept by which they performed a comparison of ISP against 
accepted standards outlined in the Simulation Management Plan, 
Cruise Missile Project Office, July 1988. The certification 
includes validation data and verification data. The 
comparisons of simulation data with flight data comprised 
validation. The comparison of simulation data with previously 
certified simulation data (from earlier ISP versions 10.0, 
11.0, & 12.03) comprised verification. In the case of ISP 
13.0 the validation is performed by comparing its data with 
selected test flight telemetry data, and verification is 
performed by comparing against MVS-RLS^° data. The 

ISP -11.0 was updated/certified under a new name, MVS 
(RLS-1.1), in December 1991. 
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Certification Test Matrix described in TABLE 5-4 
perform this function and included all the runs f 
13.0 has been subjected to during certification 
TABLE 5-4, ISP CERTIFACATION TEST MATE 


was used to 
or which ISP 
testing. 

IX. 


MISSILE 


PLIGHT 


OPS 


PLATPORM 


REMARKS ] 


109» 


OTL-39 


BDV014 


SUB-TTL 


* 


109C 


OTL-78 


LDV207 


SUB-CLS 


* 


109D 


OTL-45 


BNV153 


SHIP-VLS 


* 


109D 


DT-3 


BDV012 


SHIP-ABL 


★ 


109C 


OTL-38 


BDV014 


SHIP-ABL 


★ 


109D 


OTL-32 


BNV152 


SUB-TTL 


1 

* 


109C 


OTL-40 


BDV014 


SUB-TTL 


* 


109C 


TBAR-I 


LDV400 


SUB-CLS 




109C 


TBAR-2 


LDV400 


SHIP-ABL 


** 


109C 


TEAR -3 


LDV400 


SHIP-ABL 


1 

* * 


109D 


TEAR- 4 


LDV400 


SHIP-ABL 




,109C 


TBAR-t 


LDV400 


SHIP-VLS 


* ★ 


109D 


TEAR- 8 


LDV400 


SHIP-VLS 




109C 


DT-1 


LDV400 


SUB-TTL 




109C 


DT-IR 


LDV400 


SHIP-ABL 




109D 


DT-2 


LDV401.1 


SHIP-ABL 




109C 


DT/OT-1 


LDV401.1 


SUB-TTL 




Notes: * : 


ESP- 10.0 valJ 


Ldation runs, resulted in 


. provisional 



certification for BLK III ISP development. 



** ISP- 11.0 verification test runs, resulted in 
provisional certification for BLK III test 
flights . 

*** ISP 13.0 validation and verification runs, to be 
completed for full certification. 

The following parameters were plotted for ISP 13.0 
runs from the test matrix a±>ove and compared with flight 
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telemetry data for validation, and MVS (RLS-1.1) simulation 
for verification. The parameters were: 

• Latitude versus Longitude. 

• Inertial Altitude versus time. 

• Roll versus time. 

• Yaw versus time. 

• Roll rate versus time. 

• Pitch rate versus time. 

• Yaw rate versus time. 

• Mach command versus time. 

• Mach versus time. 

• North velocity versus time. 

• East velocity versus time. 

• Vertical velocity versus time. 

• Normal acceleration versus time. 

• TOA error versus time. 

• Air temperature versus time. 

• Static pressure versus time. 

• Dynamic pressure versus time. 

A typical plot that was developed to perform the 
comparison function for the DTIR; ISP 13 . 0 ... Inertial 
Altitude, Vertical Velocity, and Normal Acceleration vs 
Telemetry can be seen in the right hand column at Figure 11. 
The solid line in these graphs represents the simulation 
results. The dotted lines are the actual missile telemetry 
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Figure 11 Telemetry vs Simulation Comparison 
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readings overlaid on the computer simulation graph. Figure 11 
also depicts an example of the type of close match the 
Simulation Management Board would be looking for during the 
validation and certification process. A perfect match would 
be both graphs laid one upon the other. 

The closer the simulation results (dotted lines) match 
the missile telemetry data (solid lines) the more accurate the 
simulation would be to the true life event it was modeling. 
If this were true for the majority of the parameters being 
modeled, and the results were consistent over a number of runs 
(simulation matched with flight telemetry data) , then it could 
be reasonably assumed that the model was an accurate portrayal 
of these events. This is the approach that the TLAM Program 
Office is pursuing for their ISP model for the TLAM. 

While the missile system attributes may be different 
for Longbow HELLFIRE system, the basic procedure to perform 
the certification process would be the same. This process is 
one of matching a number of simulation runs with 
their corresponding flight telemetry data, and then assessing 
the results between these two runs. 

As with the TLAM Program, an independent Software 
Management Board (SMB) could be established to review the data 
and determine whether a certification of the simulation could 
be rendered. 

To implement and control this task, a weapon specific 
simulation certification plan and process would have to be 
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developed. An example of the process that TLAM used in their 
certification effort can be seen in Appendix C. This process 
could easily serve as a guideline for Longbow HELLFIRE's 
certification effort with some modifications. 

3. Summary. 

Validation and Certification of the HWIL simulation 
represents a significant challenge, but one that can be 
solved. The TLAM example is just one possible solution to 
this challenge. Further insight into this area can be 
obtained through a new book on the process of validating 
simulations entitled: "Simulation Validation, A Confidence 
Assessment Methodology". This book provides a generic 
foundations for describing, assessing, and structuring the 
simulation validation process and many of the problems areas 
that may be encountered along the way^^. 



Knepell, P.L., Simulation Validation, IEEE Computer 
Society Press, Los Allmltos, California, 1993. 
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VI CONCLUSION 



A. INTRODUCTION. 

After analyzing all the pertinent information and data 
regarding the Longbow HELLFIRE HWIL lot acceptances plan, 
several areas have been identified that may require additional 
attention prior to implementation of this test plan. These 
areas will form the basis of this chapter and will be 
discussed individually in order of importance. 

B. BASIS. 

In comming to a conclusion several areas stood out as 
being of particular importance to implementing the final HWIL 
acceptance plan. These areas are: 

1) The role of Longbow HELLFIRE missile flight testing, as 

it pertains to HWIL lot acceptance testing. 

2) The nature and extent of the validation and 

certification of the HWIL simulation. 

3) How improvements in the process will be achieved. 

4) Implications and future use by other systems. 

5) Stockpile Surveillance. 

1. The Role of Flight Testing. 

The role of Longbow HELLFIRE missile flight testing 
presents one of the biggest potential dangers to the program 
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if not addressed satisfactorily. Here you have a case where 
an insufficient quantity of missiles that will have all passed 
the HWIL simulation test are taken to the range and fired at : 

I 

a given scenario target with no attempt at either separating 
the results from the simulation or qualifying them with 
respect to the simulation. In both cases, given the small 

1 

quantity (one missile flight per quarter) it will be extremely 
difficult to draw any kind of meaningful conclusion if a 
missile should fail to hit its target. Increasing the number 
of flight tests per quarter for the first fiscal year should 
facilitate the validation of the HWIL simulation. Failure ; 
analysis would be limited to trajectory and film reviews of 

the flight, and the specter of whether or not the simulation ! 

i 

was passing on defective missiles could not be easily j 
answered. The intended role of these flights was to prove out i 
that non- tested areas of the missile such as the warhead and ; 
rocket motor will and do in fact function properly. The . 
probability of detecting faults in these non- tested areas is 
not very large even when they occur in a significant 
proportion (say 15%) of the population of missiles if only 
four missiles are sampled per year. The probability, P^j, of 
detecting such a fault (if at least one of 4 missile fail) 
when 15% of the missiles are defective is: 

= 1 - (0.85)^ = 0.478 

The niomber of tests, n, needed per year to assure that the 
estimate, R, of the true successful flight reliability R is 
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within one- fourth of a standard deviation of R with 
probability 0.90 is approximated by: 

n = (1.645)^ * 4^ = 43 

To be within one-half standard deviation of R with probability 
0.90 requires about 10 missiles per year. If R = 0.85, the 
standard deviation is (0 . 85) * (0 . 15) = 0.1275. To be within 
one- fourth a standard deviation equates to R differing from 
0.85 by + or - 0.03, with probability 0.90. To be within one- 
half a standard deviation equates to R differing from 0.85 by 
+ or - 0.06, with probability 0.90. 

2. Validation and Certification. 

Validation and certification should be concerned with 
what can be reasonably tested and substantiated within the 
HWIL simulation test and what cannot. Knowing these limits 
and limitations will help determine what modifications to the 
entire procedure are warranted. As described earlier in 
Chapter V, there are existing programs that have successfully 
validated and certified their simulations. The TLAM example 
is just one method that has been performed that fulfills this 
requirement, and could be used as a model for the Longbow 
HELLFIRE validation and certification process. 

3 . Improvements . 

It stands to reason that this is only the first step 
in what will be an eventual dependence on HWIL simulations for 
lot acceptance. Most improvements to the process will only be 
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discovered after the facility is operational and certain 
amount of experience has been gained. With this in mind, some 
attention should be given to the area of planning for process 
improvements now to more easily take advantage of better 
methods as they become available. 

4. Future System Application. 

Other Army programs have been approached about using 

the STAF facility on a time sharing basis and using HWIl] 

1 , 

simulation to reduce testing costs. This program would! 
include Non-Line of Sight (NLOS) , the earlier HELLFIRE II,' 
JAVELIN man-portable anti-tank missile. Brilliant Anti-tank 
(BAT) , and CORPS SAM. The key factor here is the need tol 
highlight the necessity of early planning to accomplish thiS' 

[ 

task. 

Longbow HELLFIRE has already demonstrated that it is 
much easier to design features into the missile that lend 
themselves to aiding the ease of HWIL testing, than attempting 
to correct these after the missile design has been locked in. 
In the case of Longbow HELLFIRE, the move to HWIL simulation 
meant determining an after the fact means to attach the 
simulation hook up cable through a sealed access panel in the 
missile body. In addition numerous signal and interface 
problems and modifications were needed. These requirements 
would have been easier to resolve if they had been included in 
the original system design to accommodate an HWIL approach. 
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Future systems should investigate this area during the design 
phase, so that HWIL critical features can be incorporated into 
the design. 

5. Stockpile Surveillance. 

Although, not a part of the Longbow HELLFIRE QALVT 
program, an area that warrants attention here because of its 
obvious impact, is stockpile surveillance. Normally at a 
predetermined time in the storage life of missiles, a certain 
quantity of missiles are selected at random from a given lot, 
inspected and test fired. This process helps assess 
deterioration characteristics of stored missiles and 
facilitate the identification of problems that can be 
corrected before they continue to degrade the rest of the 
stockpile. HWIL simulations may be useful for this type of 
testing and may provide substantial savings for the 
Government. In addition, some failures could in all 
likelihood be traced to a specific component and corrected; 
whereas with flight testing, the cause of the failure might 
never be determined. With failure modes identified, it would 
be that much easier to implement corrective fixes or 
incorporate them in future lots or designs. 
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VII AREAS OF CONCERN 



A* GENERAXi • 

The purpose of this chapter is to identify potential problem 
areas that have so far come to light in this thesis. It is ( 
hoped that these suggestions are viewed as an aid to solving 
these problems and not as unsolicited criticism to a process \ 
that is already well along its way to providing substantial | 
benefits for the Army and DOD. 

B. THE PROBLEMS AREAS. 

1. Flight Testing. j 

Flight testing, as it is currently planned in the Longbow; 
HELLFIRE program, does not appear to provide significant added 
value to the QALVT program. The original purpose of this 
testing was to provide a basis to checkout other component 
areas of the Longbow HELLFIRE missile system that were not 
tested during the HWIL simulation process. However it suffers 
from certain major deficiencies that will hamper obtaining any 
meaningful results. Because only one missile out of a 
possible 250-400 missiles produced during a quarter will be 
fired, it will be extremely hard to draw any solid conclusions 
from any failures that may occur during these test flights. 
Since these test articles will be missiles accepted straight 
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out of the STAF testing facility, failures after passing in 
those tested areas will be hard to explain if not attributed 
directly to some shortcoming in the HWIL simulation test 
itself . 

There are no easy answers to this problem. Flight 
testing is absolutely essential to the process of determining 
whether the warhead, rocket motor, safe and arming functions, 
as well as the functions that were tested via HWIL are 
functioning properly under actual operational conditions. The 
crux of this issue is how to relate these tests to the 
simulation in a meaningful manner. 

2. Feedback. 

The HWIL simulation was developed and constructed with 
the aide of an extensive amount of actual fight test data and 
simulation work within the MICOM Laboratories. Because of 
this effort the people involved were able to develop this 
process to the point that it can be used for lot acceptance 
testing. This effort has been done largely in parallel with 
the current Longbow HELLFIRE EMD program. It has taken 
advantage of the numerous instrumented missile flight tests to 
perfect the HWIL simulation process. But now as EMD ends and 
production begins, access to additional sources of future 
flight data will not be available for process improvement. As 
discussed earlier only production configured missiles will be 
flight tested in the future. This will not provide any 
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substantial feedback into the HWIL simulation process so that 
it can be further refined and improved. 

3. Utilization By Future Systems. 

As described earlier, a whole generation of future 
tactical and operational level missile systems could all 
benefit from this type of lot acceptance testing. The 
majority of these missiles already surpass the MICOM QALVT 

criteria or are very close to it. The major concern here is , 

I 

that these systems need to make decisions about simulation | 

I 

testing now while still in their developmental stages. It has I 
already been discovered that as these systems increase in 
complexity the need to design in access and compatibility with 
simulation testing up front is crucial. Delaying this ] 
activity will only create additional design, interface, and 
cost problems. 

While this is not a Longbow HELLFIRE specific problem, 
the people matrix from the Product Assurance Directorate to 
the program office should help disseminate this information to 
the rest of the community at large so that these insights can 
be incorporated into the next generation of missile systems. 
Lot acceptance planning, methods, processes, and procedures 
need to be identified and planned for in the early stages of 
development . 
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APPENDIX A 



LONGBOW HELLFIRE PROGRAM INFORMATION 



DEVELOPER : 


PEO Tactical Missiles; Project Manager, 
Air- to-Ground Missile Support Project 
Office, ATTN: SFAE-MSL-HD, Redstone 
Arsenal, AL. 35898-5610 


PHONE : 


(205) 876-8367 


CONTRACTOR (S) : 


Martin Marietta, Rockwell International, 
and Westinghouse . 


CONCEPT: 


Longbow is a MMW fire-and- forget Hellfire 
missile system. It has an improved range 
over the standard Hellfire, and adds the 
capability for all weather operations. 


TECHNOLOGY : 


MMW fire control radar and seeker. 


LETHAL 

MECHANISM: 


Shaped charge warhead. 


PRIMARY 
TARGET : 


Armored vehicles. 


PERFORMANCE : 


Classified 


STATUS : 
FUNDING : 


Currently in the Engineering and 
Manufacturing Development (EMD) phase. 


TIMETABLE : 


Production should begin in late FY94. 


SOURCE 
SUPPORT : 


Project Manager, Air- to-Ground Missile 
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APPENDIX B STAF TEST FACILITY CAPABILITIES 



OPEN LOOP TESTS 

TRANSMIT/RECEIVE CHARACTERISTICS 
VERIFY ANTENNA PATTERNS 
VERIFY SEEKER POLARIZATION 
VERIFY TRANSMITTER PULSE SHAPE 
VERIFY SEEKER POWER 

VERIFY TRANSMITTER CENTER FREQUENCY 
VERIFY TRANSMITTER PRF 
VERIFY TRANSMITTER FREQUENCY STEPS 
VERIFY TRANSMITTER BANDWIDTH 
VERIFY CROSS -POLARIZATION ISOLATION 

ANTENNA CHARACTERISTICS 

VERIFY ANTENNA PATTERNS 
VERIFY BEAM WIDTH 

VERIFY CROSS -POLARIZATION ISOLATION 
TRACKING CHARACTERISTICS 



VERIFY RANGE DISCRIMINATES 
VERIFY ANGLE DISCRIMINATES 
VERIFY DOPPLER RESOLUTION 
VERIFY RANGE RESOLUTION 

INERTIAL /CONTROL CHARACTERISTICS 

VERIFY FIN RESPONSE 
VERIFY FIN FEEDBACK 
VERIFY INERTIAL MEASUREMENT SENSOR 

CLOSED LOOP TESTS 



COMPLETE SYSTEM FUNCTIONALITY TESTS IN NONDESTRUCTIVE TACTICAL 
FLIGHT SCENARIO 

INCLUDING: 

ACTUAL TACTICAL LINE OF SIGHT RATES, 

RANGE EXTENDED TARGETS, 

TARGET DYNAMICS, 

VARIOUS LAUNCH CONDITIONS AND GEOMETRIES 
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APPENDIX C TLAM SIMULATION CERTIFICATION PROCESS 
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Figure 2 Certification Flow Chart for Effectiveness and Survivability Simulation! 

CERTTHCATTON PLAN .\PPROVAL 

Once a simulation has been authorized for Tomahawk effectiveness at 
survivability analyses the SMB tasks the TWS E&S SCP with scheduling the simulation fl 
cerdficadon/accreditation. The first step in this process is the submission of a certificadon plan 
the SCP for approval. The certification plan specifies what will be tested, how it will be teste 
and what comparisons will be made to reference check cases and other data in the process 
cerdficadon of a simuladon. A basic set of reference check cases are detailed at the end of th 
appendix. These reference check cases allow the responsible organizadon a media for comparisc' 
to demonstrate that a simuladon is certifiable. Comparison of simuladon data to reference chec 
case data should be included as the first step in any cerdficadon plaiL The responsible organizadc 
may propose any cost effective combination of the below methodologies to complere cerdficadoi 
In order of decreasing priority: 

1) Comparison to flight test data; 

2) Comparison to other simulations whose certification basis is flight teJ 
comparison; 

3) Comparison of data with other certified simulations; ! 

4) Technical audit of simuladon performance; | 

5) Demonstration of simuladon capability. 
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